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WHAT IS CLAIMED IS: 

1. A method for compositing a computer-graphics image and a picture 
taken by a camera comprising: 

5 defining a three-dimensional model, a viewpoint, and a plane of 

projection, in a space established on a computer; 

defining lines of sight extending from the viewpoint to projection pixels 
on the plane of projection so that each of the lines of sight conforms with a ray of light 
incident on a pixel corresponding thereto of the picture taken by the camera; 
10 tracing the lines of sight extending from the viewpoint through the plane 

of projection and the three-dimensional model to obtain attributes of portions of the 
three-dimensional model corresponding to the projection pixels, thereby forming a 
two-dimensional image of the three-dimensional model on the plane of projection; and 
superposing the two-dimensional image on the picture to generate a 
15 composite image. 

2. The method of claim 1, further comprising providing a calibration table 
having first data and second data correlated with each other, the first data concerning 
positions of pixels of the picture taken by the camera and the second data concerning 

20 directions and positions of rays of light incident on the pixels of the picture, wherein 
the lines of sight are defined based upon the directions and positions of the rays of 
light incident on the pixels of the picture corresponding to the projection pixels, 
obtained by looking up the second data with the first data in the calibration table. 
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3. An apparatus for compositing a computer-graphics image created by 
rendering a three-dimensional model and a picture taken by a camera, comprising: 

a calibration table storage unit for storing a calibration table having first 
data and second data correlated with each other, the first data concerning positions of 
pixels of the picture taken by the camera and the second data concerning directions 
and positions of rays of light incident on the pixels of the picture; 

a line-of-sight calculation unit for obtaining lines of sight extending 
from a viewpoint to the three-dimensional model, based upon the directions and 
positions of the rays of light incident on the pixels of the picture, obtained by looking 
up the second data with the first data in the calibration table, so that each of lines of 
sight passing through projection pixels on a plane of projection conforms with a ray of 
light incident on a pixel corresponding thereto of the picture taken by the camera; 

a two-dimensional image generation unit for generating a two- 
dimensional image on the plane of projection from the three-dimensional model by 
tracing the lines of sight so as to obtain attributes of portions of the three-dimensional 
model corresponding to the projection pixels on the plane of projection; and 

a composite image generation unit for superposing the two-dimensional 
image on the picture, to generate a composite image. 

4. The apparatus of claim 3, wherein each piece of the second data of the 
calibration table includes a direction in which a ray of light strikes on a pixel of the 
picture and a displacement from a base point to the incident light. 

5. The apparatus of claim 3, wherein one piece of the second data of the 
calibration table includes coordinates of two points on the incident light. 
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6. A program for compositing a computer-graphics image and a picture 
taken by a camera, the program capable of causing a computer to perform the steps of: 

defining a three-dimensional model, a viewpoint, and a plane of 
projection, in a space established on a computer; 

defining lines of sight extending from the viewpoint to projection pixels 
on the plane of projection so that each of the lines of sight conforms with a ray of light 
incident on a pixel corresponding thereto of the picture taken by the camera; 

tracing the lines of sight extending from the viewpoint through the plane 
of projection and the three-dimensional model to obtain attributes of portions of the 
three-dimensional model corresponding to the projection pixels, thereby forming a 
two-dimensional image of the three-dimensional model on the plane of projection; and 

superposing the two-dimensional image on the picture to generate a 
composite image. 

7. A method for rendering a three-dimensional model created by computer 
graphics into a two-dimensional image to be superposed on a picture taken by a 
camera to form a composite image, the method comprising: 

defining a viewpoint, and a plane of projection, in a space established on 
a computer where the three-dimensional model is located; 

defining lines of sight extending from the viewpoint to projection pixels 
on the plane of projection so that each of the lines of sight conforms with a ray of light 
incident on a pixel corresponding thereto of the picture taken by the camera; 

tracing the lines of sight extending from the viewpoint through the plane 
of projection and the three-dimensional model to obtain attributes of portions of the 
three-dimensional model corresponding to the projection pixels; and 
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setting the obtained attributes of the portions of the three-dimensional 
model to the projection pixels corresponding thereto, to form a two-dimensional image 
of the three-dimensional model on the plane of projection. 

8. The method of claim 7, further comprising providing a calibration table 
having first data and second data correlated with each other, the first data concerning 
positions of pixels of the picture taken by the camera and the second data concerning 
directions and positions of rays of light incident on the pixels of the picture, wherein 
the lines of sight are defined based upon the directions and positions of the rays of 
light incident on the pixels of the picture corresponding to the projection pixels, 
obtained by looking up the second data with the first data in the calibration table. 

9. An apparatus for rendering a three-dimensional model created by 
computer graphics into a two-dimensional image to be superposed on a picture taken 
by a camera to form a composite image, the apparatus comprising: 

a calibration table storage unit for storing a calibration table having first 
data and second data correlated with each other, the first data concerning positions of 
pixels of the picture taken by the camera and the second data concerning directions 
and positions of rays of light incident on the pixels of the picture; 

a line-of-sight calculation unit for obtaining lines of sight extending 
from a viewpoint to the three-dimensional model, based upon the directions and 
positions of the rays of light incident on the pixels of the picture, obtained by looking 
up the second data with the first data in the calibration table, so that each of lines of 
sight passing through projection pixels on a plane of projection conforms with a ray of 
light incident on a pixel corresponding thereto of the picture taken by the camera; and 
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a two-dimensional image generation unit for generating the two- 
dimensional image on the plane of projection from the three-dimensional model by 
tracing the lines of sight so as to obtain attributes of portions of the three-dimensional 
model corresponding to the projection pixels on the plane of projection. 

10. A program for rendering a three-dimensional model created by computer 
graphics into a two-dimensional image to be superposed on a picture taken by a 
camera to form a composite image, the program capable of causing a computer to 
perform the steps of: 

defining a viewpoint, and a plane of projection, in a space established on 
a computer where the three-dimensional model is located; 

defining lines of sight extending from the viewpoint to projection pixels 
on the plane of projection so that each of the lines of sight conforms with a ray of light 
incident on a pixel corresponding thereto of the picture taken by the camera; 

tracing the lines of sight extending from the viewpoint through the plane 
of projection and the three-dimensional model to obtain attributes of portions of the 
three-dimensional model corresponding to the projection pixels; and 

setting the obtained attributes of the portions of the three-dimensional 
model to the projection pixels corresponding thereto, to form a two-dimensional image 
of the three-dimensional model on the plane of projection. 
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